1. Technical Field
One or more embodiments relate generally to systems and methods for detecting and modifying shadows in an image. More specifically, one or more embodiments relate to systems and methods of patch-based detection and removal of cast shadows in a digital image.
2. Background and Relevant Art
Photographs often include cast shadows created by objects outside the frame of the images. For example, a cast shadow can be a product of a shadow of the person taking an image when a light source is behind the person taking the image. Cast shadows can create unwanted artifacts and/or color or luminance differences within the image that can hide or alter an appearance of objects within the frame of the image, detracting from the visual appeal of the image. In particular, the period of time just after sunrise or just before sunset provides soft, diffuse lighting that is often desirable for taking photographs, but the low altitude of the sun can produce long shadows that interfere with the image.
Image processing techniques that allow for the removal or other modification of cast shadows from an image can be useful in generating an image that simulates what the image would look like without the shadow. Unfortunately, conventional image processing systems that allow for the removal of shadows have several drawbacks. For instance, conventional image-processing systems often have difficulty in identifying cast shadows. In particular, the variations in texture and other properties of different materials visible within an image can make it difficult to accurately predict where shadows are in an image. In addition, conventional image-processing systems often have difficulty to distinguishing between shadows cast by objects that are within the image and shadows cast by objects that are outside the image. Due to the ambiguous nature of shadow detection, many conventional image-processing systems rely on manual input to initially identify shadows in an image. Manually identifying the shadows can be tedious, as well as imprecise or variable based on the perception of the user providing the input.
Some conventional image-processing systems are proficient at detecting shadows with hard edges using illuminant invariant features of an image (e.g., hue channel in the HSV color space). Because shadow boundaries do not exist in such illuminant invariant features, conventional image-processing systems can detect shadows with hard edges. Such image-processing systems, however, often do not handle soft shadow boundaries well and can produce visible artifacts along the boundaries as a result. Other image-processing systems attempt to overcome deficiencies with handling soft shadows by estimating shadow density directly using patch lightness and applying color adjustments for each penumbra region. While such methods are able to identify soft shadows in images or shadowed portions that primarily include the same texture, estimating shadow density in this manner may produce poor results for images that contain multiple textures or that include different textures within the shadowed regions.
Even if a shadow region is previously identified, manually or otherwise, shadow removal is nonetheless challenging. In particular, simple color and intensity correction of shadowed pixels does not usually produce good results in the presence of soft shadows and/or complex spatially-varying textures. As such, many conventional image-processing systems are not able to produce high quality shadow removal on anything but simple scenes.
These and other disadvantages may exist with respect to identification and removal of shadows from digital images.